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CHAPTER  I 


INTRODUCTION 

Statement  of  the  Problem 

*Uf 

The  Armed  Services  Procurement  Regulation  (ASPR)  , 
Section  1-705.4,  states  that  the  Small  Business  Administra- 
tion (SBA)  "has  statutory  authority  to  certify  the  compe- 
tency of  any  small  business  concern  as  to  all  elements  of 
responsibility  . . . £ 18 1 p, It  124]."  If  a contracting  offi- 
cer decides  that  the  bid  of  a small  business  concern,  which 
Is  otherwise  the  most  acceptable,  should  be  rejected  because 
that  concern  Is  nonresponsible,  the  SBA  may  issue  the  con- 
cern a certificate  of  competency  (COC).  The  contracting 
officer  must  then  either  make  the  award  or  appeal  the  deci- 
sion through  SBA  channels.  The  final  decision  is  made  by 
the  SBA  Associate  Administrator  for  Procurement  Assistance 
and  Is  binding  on  the  contracting  office.  Although  final 
decisions  on  competency  are  made  by  the  SBA,  it  has  no 
stated  responsibility  for  the  performance  of  the  certified 
business.  These  procedures  lead  to  the  potential  for  exces- 
sive costs  to  the  Air  Force  due  to  contractor  default  or 
untimely  deliveries.  Consequently,  a reed  exists  to 

*The  ASPR  became  the  Defense  Acquisition  Regula- 
tion (DAR)  effective  1 June  1978. 
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determine  if  certificate  of  competency  issuance  has  an 
adverse  Impact  on  Air  Force  contracting. 

Scope 

During  the  period  covering  Fiscal  Years  (FY)  1973- 
78,  the  Department  of  Defense  awarded  approximately  20  per- 
cent of  its  contracts,  in  dollar  value,  to  small  business 
firms.  For  FY  1978,  the  amount  was  about  $10.7  billion. 

The  FY  1978  total  for  the  Air  Force  reached  nearly  $1.9  bil- 
lion. Over  one- third  of  the  Air  Force  total  awards  to 
small  business  were  made  by  the  five  Air  Logistics  Centers 
(ALCs)  under  the  Air  Force  Logistics  Command  (5).  Since 
ALC  contracting  with  small  business  represents  a substantial 
portion  of  that  done  in  the  Air  Force,  the  study  concen- 
trates on  ALC  contracting  data  relative  to  COC  issuance. 

Definition  of  Terms 

Small  Business  Concern.  For  the  purpose  of  this 
study,  a small  business  concern  shall  be  a concern  that  is 
independently  owned  and  operated  and  is  not  dominant  in  the 
field  of  operation  in  which  it  is  bidding  on  government 
contracts  (20ip.lil03). 

Responsible  Contractor.  For  the  purpose  of  this 
study,  a responsible  contractor  is  a contractor  that  meets 
the  following  minimum  standards! 
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1.  "Have  adequate  financial  resources  or  the 
ability  to  obtain  such  resources  as  required  during  the 
performance  of  the  contract  . . . [20ip.lil59]." 

2.  "Be  able  to  comply  with  the  required  or  proposed 
delivery  or  performance  schedule,  taking  into  consideration 
all  existing  business  commitments  . . . [20  ip. It  159], " 

3.  "Have  a satisfactory  record  of  performance  . . , 
[20»p.l«160]." 

4.  "Have  a satisfactory  record  of  integrity  . . . 
[20ip.lil60]." 

5.  "Be  otherwise  qualified  and  eligible  to  receive 
an  award  under  applicable  laws  and  regulations  . . . 
[20«p.l«160]." 

Capacity  and  Credit.  Category  of  responsibility 
standards  that  comprise  those  issues  which  involve  the  con- 
tractor's ability  to  perform  the  work  called  for  by  the 
contract.  Capacity  implies  that  the  contractor  has  the 
facilities  and  equipment  or  can  obtain  the  facilities  and 
equipment  in  time  to  perform  the  contract.  Credit  implies 
that  the  contractor  has  the  financial  resources  or  can 
obtain  the  financial  resources  required  for  the  performance 
of  the  contract  (6»187). 

Tenacity  and  Perseverance.  Category  of  responsi- 
bility standards  that  comprise  those  issues  which  involve 
the  contractor's  will  to  perform  the  work  called  for  by  the 
contract  ( 6 i 187) . 
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Integrity.  For  the  purpose  of  this  study.  Integrity 
Implies  the  will  of  the  contractor  to  adhere  to  a code  of 
values  of  honesty  and  uprightness  (6tl86). 

Pre -Award  Survey.  An  evaluation,  by  a contracting 
officer  or  his  representative,  of  a prospective  contractor's 
operation  to  determine  if  the  contractor  Is  responsible  and 
can  perform  under  the  terms  and  conditions  of  the  proposed 
contract  (20ip.l»160) . 

Certificate  of  Competency.  Certificate,  issued  by 
the  Small  Business  Administration  to  a small  business,  cer- 
tifying that  the  small  business  is  a responsible  contractor 
for  the  performance  of  a specific  government  contract 
(20ip,lil24) . 

Delinquent  Delivery.  Actual  failure  by  the  con- 
tractor, regardless  of  reason,  to  meet  the  contract  delivery 
or  performance  schedule  (20ip.25i2). 

Termination  for  Default.  "...  the  exercise  of  a 
contractual  right  of  the  Government  to  terminate  the  con- 
tract in  whole  or  in  part  by  reason  of  the  contractor's 
failure,  actual  or  anticipatory,  to  perform  his  obligations 
under  the  contract  £20ip.8i33]." 

Research  Objective 

The  objective  of  this  research  is  to  examine  con- 
tract information  from  the  five  ALCs  to  determine  if  COC 
procedures  have  an  adverse  impact  on  ALC  contracting  with 
respect  to  contractor  performance. 


Research  Questions 

In  order  to  satisfy  this  objective,  the  research 
questions  to  be  answered  arei 

1,  To  what  extent  has  poor  contractor  performance, 
with  respect  to  late  deliveries  and  contractor  default, 
been  evident  In  cases  In  which  COCs  have  been  Issued? 

2,  To  what  extent  has  poor  contractor  performance, 
with  respect  to  late  deliveries  and  contractor  default, 
been  evident  In  cases  In  which  referrals  to  the  SBA  were 
made,  but  no  COCs  were  issued? 

3,  Is  there  a significant  difference  In  contractor 
performance  under  each  of  the  two  categories  mentioned 
above? 

Organization  of  the  Study 

The  following  chapter  organization  has  been  selected 
for  the  presentation  of  this  study.  Chapter  II  covers  a 
background  review  of  the  SBA  and  its  Certificate  of  Compe- 
tency Program.  Chapter  III,  Methodology,  then  discusses  the 
methods  chosen  for  Identifying,  selecting,  and  testing 
appropriate  data  in  order  to  answer  the  research  questions. 
Chapter  IV  presents  the  data  collected  and  states  the  find- 
ings resulting  from  a statistical  analysis  of  the  data. 
Finally,  Chapter  V presents  a summary  of  the  research  accom- 
plished, describes  the  conclusions  drawn  from  the  findings, 
and  offers  recommendations  for  further  research. 
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CHAPTER  II 

A REVIEW  OF  THE  SBA  AND  ITS  CERTIFICATE  OF 
COMPETENCY  PROGRAM 


Importance  of  Small  Business 

A widely  accepted  view  that  underlies  our  economic 
and  political  system  is  that  it  is  in  the  national  interest 
of  our  society  to  have  numerous,  economically  significant 
and  effective  small  businesses.  Only  through  the  establish- 
ment of  a large  number  of  small  businesses  will  our  society 
be  assured  of  maintaining  the  competition  that  is  requisite 
of  a free  enterprise  system  (8»vij  11»9). 

The  American  principle  of  maintaining  a society 
based  upon  the  free  enterprise  system  was  felt  to  be  of  such 
Importance  that  Congress,  in  writing  the  Small  Business  Act 
of  1953,  statedt 

The  essence  of  the  American  economic  system  of  pri- 
vate enterprise  is  free  competition  ....  The  pre- 
servation and  expansion  of  such  competition  is  basic 
not  only  to  the  economic  well-being  but  to  the  security 
of  this  nation.  Such  security  and  well-being  cannot  be 
realized  unless  the  actual  and  potential  capacity  of 
small  business  Is  encouraged  and  developed  (,16«232J. 

Section  3 of  the  Small  Business  Act  goes  on  to  pro- 
vide a general  definition  of  a small  business  as  being  "one 
which  is  independently  owned  and  operated  and  which  is  not 
dominant  in  its  field  of  operation  £3i237]." 


Today,  approximately  97  percent  of  all  businesses  In 
the  United  States  are  classified  as  small  businesses.  These 
10  million  small  businesses  produce  approximately  48  percent 
of  the  nation's  gross  national  product  and  provide  for  about 
55  percent  of  the  private  sector  work  force  (11i4j  15il6). 

Because  small  businesses  provide  such  a substantial 
Influence  upon  the  American  economic  system,  It  Is  readily 
apparent  that  It  should  be  in  the  national  Interest  to  pre- 
serve a strong,  dynamic  small  business  sector  within  the 
economy. 


Government's  Responsibility 

Congress  felt  that  what  was  needed  to  meet  this 
objective  was  a government  agency  that  would  be  active  In 
promoting  and  protecting  the  interests  of  the  small 
businessman  (8ivii).  An  agency  that  would 

. . • first,  provide  an  organization  to  make  small 
business  concerns  sound  and  self-sufficient  .... 
Second,  to  insure  the  participation  of  small  business 
enterprises  in  war,  or  mobilization  of  our  economy  for 
war,  and  for  peace  as  well.  Third,  ...  to  concentrate 
the  responsibility  for  administering  one  program  to 
preserve  and  develop  small  business  In  one  independent 
agency  of  the  Government  j.2il3]. 

By  passing  the  Small  Business  Act  of  1953,  Congress  estab- 
lished just  such  an  agency,  the  Small  Business  Administra- 
tion, and  declared 

. . . that  the  Government  should  aid,  counsel, 
assist,  and  protect,  . . . the  interests  of  small  busi- 
ness concerns  in  order  to  preserve  free  competitive 
enterprise,  to  insure  that  a fair  proportion  of  the 
total  purchases  and  contracts  . . . for  property  and 
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services  for  the  Government^.  . . be  placed  with  small 
business  enterprises  . . . £l6i232j. 

The  overall  mission  with  which  the  SBA  was  to  con- 
cern Itself  In  helping  small  business  can  be  described  as 
follows  (2 i 167 | 12 i 7) i 

1.  To  serve  as  a financial  Institution  by  making 
equity  capital  and  general  credit  available  to  the  small 
business  enterprise  system,  either  directly  or  in  coopera- 
tion with  banks  and  other  lending  institutions. 

2.  To  assure  that  small  businesses  receive  a fair 
and  equitable  share  of  the  federal  dollar  spent  on  goods 
and  services  required  in  government  operations. 

3.  To  provide  substantial  indirect  assistance  to 
small  businesses  through  its  management  and  technical 
assistance  programs  and  through  its  advocacy  rule,  by  which 
it  promotes  interests  of  small  businesses  to  assure  their 
general  welfare. 

4.  To  assist  members  of  socially  and  economically 
disadvantaged  groups  to  achieve  the  same  benefits  from  the 
American  system  that  are  available  to  those  that  represent 
the  majority  of  the  nation's  population. 

Organization  of  the  SBA 

The  Small  Business  Administration  is  part  of  the 
Executive  Branch  of  the  U.S.  Government.  It  is  an  inde- 
pendent agency  whose  management  is  vested  in  an 
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through  a single  administrator,  acts  as  a direct  voice  to 
the  President  for  its  clientele,  small  business. 

Internally,  the  SBA  is  functionally  organized 
around  its  principal  line  functions  of  financial,  invest- 
ment, procurement,  technical  and  management  assistance,  and 
minority  small  business  assistance  (Figure  1)  (3»5|  7.625- 
630 i 14.15). 


ASSISTANT  ADMINISTRATORS 
Figure  1 

SBA  Headquarters  Organization  (14.15) 

Through  this  type  of  organizational  structure  and 
its  various  programs,  the  SBA  attempts  to  accomplish  its 
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overall  goal  of  aiding  and  protecting  the  interests  of  small 
business  concerns.  The  Procurement  Assistance  Program  is 
one  such  program  the  SBA  has  implemented  in  attempting  to 
achieve  this  goal. 


I I 

I - 

* 


i I 


I I 

I I 
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SBA  Procurement  Assistance  Program 

The  U.S.  Government  is  the  largest  single  purchaser 
of  goods  and  services  in  America  (8i66).  In  1972,  a Fed* 
eral  Commission  investigating  government  procurement  esti- 
mated that  government  contracts  for  goods  and  services  in 
Fiscal  Year  1972  totaled  approximately  $57.5  billion 
(12*348).  By  1978,  that  market  was  estimated  to  be  over 
$70  billion  (14*1). 

Small  businesses  are  often  at  a disadvantage  in 
competing  with  larger  firms.  Several  reasons  for  this  are 
that  smaller  companies  are  limited  in  the  size  of  contracts 
they  can  handle j they  lack  easy  access  to  technical  skills 
which  a large  company  can  obtain  from  its  own  staff*  they 
are  less  able  to  digest  all  of  the  available  information 
and  keep  up  with  technical  advances*  and  their  search  for 
new  markets  is  more  confined  than  it  would  be  if  they  had 
a larger  sales  force  (15*36). 

Because  of  these  disadvantages,  there  was  a general 
belief  that  small  business  was  not  receiving  its  fair  share 
of  Federal  procurements.  Congress  also  believed  that  a 
continuance  of  this  Inequitable  situation  would  "lead  to  a 


lessening  of  free  competition,  a further  concentration  of 
economic  power,  and  a weakening  of  the  private  enterprise 
system  C8»128^." 


What  the  Federal  Government  needed  was  a purchasing 
policy  which?  on  a national  scale,  would  foster  the  main* 
tenance  of  a strong,  broad-based  business  enterprise  system 
(10il2). 

By  strengthening  the  effectiveness  of  small  busi- 
ness in  ways  that  increase  its  independence  from  Gov- 
ernment, we  can  make  sure  of  the  supremacy  of  American 
free  enterprise  in  an  economically  troubled  and  regi- 
mented world  £l5«36j. 

The  Small  Business  Administration  was  tasked  by 
Congress,  through  Section  2 of  the  Small  Business  Act,  to 
develop  policies  and  procedures  that  would  insure  that 
small  businesses  received  their  fair  share  of  government 
procurements.  The  method  established  by  the  SBA  to  carry 
out  this  responsibility  led  to  the  development  of  its  Pro- 
curement Assistance  Program. 

Organization  for  Procurement 
Assistance 

The  organization  for  carrying  out  the  policies  and 
programs  of  the  Procurement  Assistance  Program  consists  of 
a headquarters  staff  element  under  the  SBA  Associate  Admin- 
istrator for  Procurement  Assistance  in  Washington  and  field 
procurement  assistance  personnel  assigned  to  the  SBA 
regional  field  offices,  A procurement  assistance  organiza- 
tion chart  is  shown  in  Figure  2. 
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Figure  2 

SBA  Procurement  Assistance  Organization  (I4il6) 
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At  headquarters  level,  policy  development  and  pro- 
gram oversight  are  carried  out  by  the  Office  of  Procurement 
and  Technical  Assistance  (OPTA),  directly  under  the  Asso- 
ciate Administrator  for  Procurement  Assistance.  Divisions 
under  OPTA  are  organized  to  deal  with  specific  procurement 
assistance  programs.  OPTA*s  functions  are  primarily  those 
of  policy-making  and  liaison  between  the  field  offices  and 
the  government  procuring  agencies  (12*173) . 

Field  procurement  assistance  activities  consist  of 
a staff  of  procurement  specialists  assigned  to  each  of  the 
ten  Regional  Directors.  Each  of  these  procurement  staffs  is 
headed  by  an  Assistant  Regional  Director  for  Procurement 
Assistance.  The  size  of  these  regional  procurement  activ- 
ities varies  from  a high  of  22  personnel  in  Region  III 
(Philadelphia)  to  a low  of  three  in  Region  X (Seattle). 

The  size  in  most  other  regions  is  about  halfway  between 
these  two  extremes  (14*21-29). 

There  are  five  different  kinds  of  procurement 
assistance  specialists  assigned  to  the  regional  offices. 

Each  deals  with  a specific  aspect  of  small  business  pro- 
curement aid,  such  as  prime  contracting,  subcontracting,  or 
certificate  of  competency  assistance.  The  key  specialist 
is  called  the  procurement  center  representative  (PCR) 
(1*352).  Unlike  the  other  specialists,  PCRs  do  not  work  in 
the  regional  field  offices.  Rather,  they  work  at  key 
Defense  and  Federal  agency  installations  within  the  region. 
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PCRs  perform  the  dual  task  of  monitoring  the  procurement 
plans  of  the  agencies  at  the  installations  to  which  they 
are  assigned  and  seeking  to  increase  small  business  parti - 
dpation  in  them,  plus  counseling  area  small  business  firms 
on  ways  in  which  they  can  effectively  compete  for  these 
procurements  ( 14 i 17) , 

Both  the  headquarters  and  field  procurement  assist- 
ance staffs  are  organized  in  terms  of  the  specific  programs 
designed  to  assist  small  business  in  securing  government 
contracts.  The  major  programs  around  which  the  Procure- 
ment Assistance  Program  is  organized  aret  the  Prime  Con- 
tracts Program,  the  Subcontracts  Program,  the  Technical 
Assistance  Program,  and  the  Certificate  of  Competency  Pro- 
gram (13 i 14-15).  The  program  upon  which  this  study  is 
focused  is  that  of  certificates  of  competency. 

Certificate  of  Competency  Program 

The  Certificate  of  Competency  (COC)  Program  is  one 
of  the  primary  means  through  which  the  SBA  helps  small 
businesses  obtain  government  contracts.  It  does  this  by 
allowing  a small  business  firm  the  opportunity  to  appeal  a 
decision  by  a government  contracting  officer  that  the  firm, 
although  it  is  low  bidder  on  a contract,  is  not  capable  of 
performing  the  contract  in  a satisfactory  manner. 

Currently,  the  SBA  is  empowered  by  Section  8(b)(7) 
of  the  Small  Business  Act  of  1953,  15  U.S.C.  637(b)(7)i 


14 


'1 


(7) (A)  To  certify  to  Government  procurement 
officers  ...  with  respect  to  all  elements  of  respon- 
sibility! including!  but  not  limited  to(  capability! 
competency,  capacity,  credit,  integrity,  perseverance, 
and  tenacity,  of  any  small  business  concern  or  group  of 
such  concerns  to  receive  and  perform  a specific  Govern- 
ment contract.  A Government  procurement  officer  • . . 
may  not,  for  any  reason  specified  in  the  preceding  sen- 
tence, preclude  any  small  business  concern  or  group  of 
such  concerns  from  being  awarded  such  contract  without 
referring  the  matter  for  final  disposition  to  the 
Administration  [l6i238j. 

The  Act  goes  on  to  say  that-- 

(7)(C)  In  any  case  in  which  the  small  business  con- 
cern or  group  of  such  concerns  has  been  certified  by 
the  Administration  . . . to  be  a responsible  or  eligible 
Government  contractor  as  to  a specific  Government  con- 
tract, the  officers  of  the  Government  having  procure- 
ment . . . powers  are  directed  to  accept  such  certifica- 
tion as  conclusive,  and  shall  let  such  Government  con- 
tract to  such  concern  or  group  of  concerns  without 
requiring  it  to  meet  any  other  requirements  of  responsi- 
bility or  eligibility  £ 16*238]. 

The  provisions  of  this  portion  of  the  Small  Busi- 
ness Act  of  1953  are  set  forth  for  Department  of  Defense 
components  and  agencies  in  the  Armed  Services  Procurement 
Regulation  (ASPR),  Section  1-705.4.  ASPR  Section  1-705.4, 
Certificates  of  Competency,  is  divided  into  eight  sub- 
sections, (a)  through  (h),  each  of  which  deals  with  a major 
* 

aspect  of  the  program. 

Subsection  (a)  summarizes  the  main  points  of  the 
Small  Business  Act  provisions  quoted  above.  It  relates  the 
SBA's  authority  to  certify  the  competency  of  any  small  busi- 
ness with  regard  to  ".  . . capacity,  credit,  integrity,  per- 
severance, and  tenacity. " It  also  relates  the  contracting 
officer's  responsibility  to  accept  the  SBA's  decision  as 


conclusive  if  the  SBA  certifies  that  the  small  business  is 
competent  to  perform  the  contract  "unless  the  contracting 
officer  has  substantial  doubt  as  to  the  concern's  ability 
to  perform  C^O  ip.li 124] . " Appeal  procedures  for  these 
cases  are  outlined  in  later  subsections. 

Subsection  (b)  deals  with  the  special  case  of  those 
requirements  for  which  the  "highest  competence  available" 
or  the  "best  scientific  approach"  are  necessary,  such  as  in 
some  selected  research  and  development  or  for  highly  tech- 
nical products.  The  ASPR  states  that  COC  procedures  need 
not  be  applied  in  these  cases,  except  in  the  instance  where 
the  firm  that  is  judged  the  most  highly  competent  or  judged 
to  have  the  best  scientific  approach  is  denied  contract 
award  because  it  is  determined  to  be  nonresponsible.  In 
these  cases,  the  COC  process  does  apply  (20 ip.lt 124) . 

The  basic  procedures  for  the  referral  of  a bid 
rejection  to  the  SBA  are  covered  in  Subsection  ( c) . First 
of  all,  if  a Defense  contracting  officer  determines  that 
the  low  bidder  on  a particular  contract  is  nonresponsible, 
the  contracting  officer  must  notify  the  appropriate  SBA 
office  (20tp.ltl24) . 

There  are  11  SBA  offices  around  the  country  which 
are  authorized  to  process  COCs.  These  are  comprised  of  the 
10  SBA  regional  offices  identified  in  Figure  2,  plus  the 
SBA  district  office  in  Los  Angeles.  Each  of  these  11  offices 
has  a staff  of  COC  specialists  to  process  the  cases  (Ii353), 


Specific  provisions  regarding  referrals  to  SBA 
include  the  following  (20»p.lil24-6) i 

a.  The  value  of  the  contract  must  exceed  $10,000. 

b.  The  decision  by  the  contracting  officer  that 
the  low  bidder  is  nonresponsible  must  be  based  on  the 
results  of  a pre-award  survey. 

c.  The  contracting  officer  cannot  make  a concurrent 
referral  of  two  or  more  bids  on  the  same  contract.  The  SBA 
must  make  a final  decision  on  the  first  referral  before  a 
second  can  be  investigated.  If,  of  course,  the  first  bid- 
der referred  to  the  SBA  is  certified  as  competent  to  perform 
the  contract,  the  contract  must  be  awarded  to  that  bidder. 

d.  The  SBA  office  which  is  processing  the  case 
must  make  its  decision  on  the  competency  of  the  firm  within 
15  working  days  after  the  referral  is  made.  It  should  be 
noted  here  that  if  the  decision  on  issuance  of  a COC  is 
negative,  the  regional  office  processing  the  case  will  so 
inform  the  contracting  officer.  If  the  decision  is  made 

to  issue  a COC,  and  the  value  of  the  contract  does  not 
exceed  $500,000,  the  COC  can  be  issued  by  the  regional 
office  director.  If,  however,  the  decision  is  affirmative 
and  the  contract  value  exceeds  $500,000,  the  matter  must  be 
referred  to  the  SBA  headquarters  office  in  Washington,  D.C. 
for  final  approval.  This  approval  rests  with  the  SBA 
Associate  Administrator  for  Procurement  Assistance  (It 354). 
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e.  A referral  cannot  be  made  to  the  SBA  until  all 
bids  have  been  evaluated  by  the  contracting  officer. 

f.  After  referring  a case  to  an  appropriate  SBA 
office,  the  contracting  officer  must  maintain  close  contact 
with  that  office  to  insure  that  the  SBA  is  taking  action  on 
the  matter, 

g.  The  contracting  officer  need  not  refer  a case 
to  the  SBA  if  he  certifies  in  writing  that  the  urgency  of 
the  procurement  is  such  that  the  SBA  decision  process  on 
certification  of  competency  would  preclude  the  timely  pur- 
chase  of  the  contract  items. 

h.  If  the  contracting  officer  determines  that  a 
low  bidder  is  nonresponsible  and  refers  the  case  to  the 
SBA,  he  must  be  able  to  support  his  decision  by  "substan- 
tial evidence  documented  in  the  contract  files." 

Subsection  (d)  outlines  DoD's  desire  to  maintain  a 
cooperative  working  arrangement  with  the  SBA  regarding  COC 
matters.  To  assist  in  reaching  an  agreement  with  the  SBA 
on  each  referral,  the  contrapting  officer  must  furnish  cer- 
tain documentation  to  the  SBA  office  processing  the  case. 


*A  Hq  USAF/RDC  message,  292100Z  AUG  78,  to  all  Air 
Force  major  commands,  pointed  out  the  Comptroller  General's 
concern  over  whether  or  not  the  urgency  exception  was  in 
conflict  with  the  Small  Business  Act's  direction  that  all 
cases  involving  the  determination  of  nonresponsibility  for 
small  business  concerns  be  referred  to  the  SBA.  As  a result 
of  this  concern,  the  message  directed  the  major  commands  to 
refer  all  cases  to  the  SBA  and  not  apply  the  urgency  excep- 
tion. 


■ 
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This  documentation  Includes  copies  of  the  solicitation  for 
bids,  drawings  and  specifications  of  the  required  items,  the 
pre-award  survey  findings,  and  any  other  information  which 
was  pertinent  to  the  contracting  officer's  decision  that  the 
low-bidding  firm  was  nonresponsible  for  performing  the  con- 
tract (20ip.l»127) , 

If  the  SBA  plans  to  certify  the  competency  of  a 
small  business  judged  nonresponsible  by  a Defense  contract- 
ing officer,  it  must  provide  the  reasons,  in  writing,  to 
the  contracting  officer  and  allow  him  an  opportunity  to 
present  new  or  additional  information  on  the  case  before 
the  final  decision  is  made.  These  provisions  are  detailed 
in  Subsection  (e) . The  ASPR  here  states  that  "every  effort 
should  be  made  to  resolve  any  differences  between  the  SBA 
and  the  Departments  through  a complete  exchange  of  pre-award 
information  developed  by  each  agency  [20ip.lil27], " 

Subsection  (f)  relates  the  possible  courses  of 
action  that  can  occur  following  the  exchange  of  information 
under  Subsection  (e).  These  are  that  (20ip.lil27-9) i 

a.  The  new  and  additional  facts  provided  by  the 
SBA  are  sufficient  to  persuade  the  contracting  officer  that 
his  decision  that  the  low  bidder  was  nonresponsible  should 
be  reversed  and  the  referral  to  the  SBA  should  be  withdrawn. 
In  these  cases,  the  contract  will  be  awarded  without  the 
issuance  of  a COC. 
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b.  The  SBA  regional  office  and  the  Defense  con- 
tracting officer  cannot  agree  and  the  contracting  officer 
still  has  serious  doubts  about  the  low  bidder's  capability 
to  perform.  In  this  case,  the  contracting  officer  can 
request  that  the  SBA  regional  office  curtail  action  and 
forward  the  case  to  Washington,  D.C.,  for  review  by  SBA 
headquarters . 

(1)  At  the  headquarters,  the  case  is  reviewed 
by  the  Central  Office  COC  Review  Committee,  which  is  com- 
prised of  legal,  financial,  and  technical  experts,  and  a 
recommendation  is  made  to  the  Associate  Administrator  for 
Procurement  Assistance  (1i35A). 

(2)  Prior  to  presentation  to  SBA  headquarters, 
however,  the  Service  involved  will  provide  all  documenta- 
tion for  review  by  DoD  officials  at  the  Assistant  Secretary 
or  Director,  Defense  Logistics  Agency,  level.  This  review 
may  result  in  a decision  to  make  the  award  and  not  forward 
the  matter  to  SBA  headquarters.  In  this  case,  the  SBA 
regional  office  will  be  informed  that  the  contract  is  being 
awarded  and  a COC  will  not  be  required. 

(3)  If  the  case  goes  to  the  SBA  Associate  Admin- 
istrator for  Procurement  Assistance  and  his  final  decision 
is  to  issue  a COC,  the  contract  will  then  be  awarded  to  the 
firm  without  any  further  requirements  of  responsibility 
being  placed  against  the  firm.  In  other  words,  the  Associate 
Administrator's  decision  is  the  final  word. 
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c.  The  contracting  officer  and  the  SBA  regional 
office  do  not  reach  agreement,  but  the  contracting  officer 
feels  "it  would  not  be  practicable  to  appeal  the  case  . . . 
nor  would  it  be  appropriate  to  withdraw  ...”  the  referral 
to  the  SBA.  In  this  case,  the  contracting  officer  tells 
the  SBA  regional  office  that  a COC  must  be  issued  before  the 
contract  will  be  awarded  to  the  low  bidder. 

Subsection  (g)  states  that  if  the  SBA  regional 
office  Issues  a COC  without  first  giving  the  contracting 
office  a chance  to  appeal  to  a higher  level,  the  contract- 
ing officer  has  the  right  to  appeal,  as  in  Subsection  (f), 
above,  after  the  formal  issuance  of  a COC,  if  he  still  has 
doubts  about  the  competency  of  the  firm  to  satisfy  the 
contract  (20 tp.lt 129) . 

Finally,  Subsection  (h)  discusses  the  case  of  the 
SBA  making  a decision  on  COC  issuance  on  the  basis  of  some 
standard  other  than  that  which  was  used  in  the  contracting 
officer's  decision  to  refer  the  case  to  the  SBA.  If  the 
SBA  declines  to  issue  a COC  based  on  a different  standard, 
it  must  notify  the  contracting  officer  of  the  fact  and  the 
reason  for  its  decision  to  do  so.  The  contracting  officer 
can  then  use  this  information  in  deciding  on  the  contract 
award. 

Summary 

It  is  widely  accepted  that  the  maintenance  of  a 
strong  and  viable  small  business  sector  in  our  economy  is 
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vital  to  our  national  Interest.  Congress  expressed  this 
view  In  passing  the  Small  Business  Act  of  1953.  The  Act 
created  the  Small  Business  Administration,  whose  overall 
goal  Is  to  actively  promote  and  protect  the  Interests  of 
small  business.  One  of  the  primary  objectives  of  the  SBA 
Is  to  assure  that  small  businesses  receive  a fair  and 
equitable  share  of  government  contracts. 

The  SBA*s  Procurement  Assistance  Program  Is 
designed  to  help  Insure  that  small  businesses  receive  their 
fair  share  of  these  government  contracts.  One  of  the  pri- 
mary subprograms  used  by  the  SBA  to  meet  this  objective  Is 
the  Certificate  of  Competency  (COC)  Program.  The  COC  Pro- 
gram does  this  by  giving  a small  business  concern  the  oppor- 
tunity to  reverse  a decision  by  a government  contracting 
officer  that  the  concern  Is  not  competent  to  complete  a 
particular  contract. 

The  primary  provisions  of  the  program,  which  are 
detailed  for  Defense  components  and  agencies  in  the  Armed 
Services  Procurement  Regulation,  Section  1-705.4,  are 

(1)  that  any  small  business  which  Is  low  bidder  on  a con- 
tract and  Is  determined  by  the  contracting  officer  to  be 
nonresponslble  may  appeal  the  decision  to  the  SBA)  and 

(2)  that  If  the  SBA  rules  that  the  firm  is  capable  of  com- 
pleting the  contract,  the  contract  must  be  awarded  to  the 
firm. 


The  following  chapter  will  present  the  methodology 
used  to  answer  the  research  questions  posed  In  this  study. 


i 
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CHAPTER  III 


METHODOLOGY 

This  chapter  describes  the  universe,  population, 
and  sample  for  this  studyi  presents  the  data  collection 
plant  and  states  the  statistical  tests  to  be  employed  in 
evaluating  the  findings. 

Universe  Description 

The  universe  under  consideration  in  this  study 
consists  of  all  contract  proposals  referred  to  the  SBA  for 
COC  action  by  Air  Force  contracting  officers.  During  Fiscal 
Years  1977  and  1978,  a total  of  286  referrals  were  made  by 
Air  Force  contracting  officers  to  the  SBA  (Table  1), 

Population  and  Sample 

Description.  The  population  under  consideration  was  limited 
to  contract  proposals  referred  to  the  SBA  for  COC  action  by 
AFLC*s  five  Air  Logistics  Centers. 

In  order  to  answer  the  research  questions,  the  pop- 
ulation of  contract  referrals  within  AFLC  was  separated 
into  two  sub -population  groups.  Contracts  awarded  as  a 
result  of  COC  issuance  comprised  one  group.  Those  contract 
proposals  referred  to  the  SBA  for  COC  action,  but  which 
were  awarded  without  Issuance  of  a COC,  comprised  the  second 


' 
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Table  1 


! 


Contract  Proposals  Referred  to  the  SBA  for  COC  Action 
During  FY  77-78--A  Breakdown  by  Commands 


Command 

FY  77 

FY  78 

AU 

0 

0 

AAC 

0 

0 

ADCOM 

1 

0 

ATC 

3 

8 

MAC 

9 

7 

SAC 

9 

9 

TAC 

1 

6 

AFCS 

2 

0 

AFLC 

90 

122 

AFSC 

9 

8 

AFAFC 

0 

0 

AFRES 

0 

0 

USAFA 

0 

1 

USAFSS 

0 

0 

PACAF 

0 

1 

USAFE 

0 

0 

TOTALS 

124 

162 

Sources  Certificate  of  Competency  Report,  RCSs 
DD-I  & L (Q)  1152  HQ  USAF/RDC,  Washington,  D.C. 
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group.  In  other  words,  the  second  sub -population  group 
consisted  of  cases  in  which  the  lowest  bidder  either 
declined  to  file  for  a COC,  withdrew  his  request  for  a COC 
once  an  investigation  for  COC  issuance  had  been  initiated, 
or  had  been  declined  issuance  of  a COC  by  the  SBA.  As  a 
result,  the  contract  was  awarded  to  the  second  lowest  bidder 
who,  for  inclusion  in  the  second  sub-population  group,  was 
determined  by  the  contracting  officer  to  be  sufficiently 
competent  to  perform  the  contract  without  COC  action. 

An  initial  survey  conducted  at  HQ  AFLC  indicated 
that  275  referrals  were  made  to  the  SBA  from  FY  76  through 
FY  78.  Of  these  275  referrals,  70  contracts  were  awarded 
after  issuance  of  a COC,  10  contracts  were  cancelled,  and 
24  were  unresolved,  resulting  in  17 1 contracts  being  awarded 
without  issuance  of  a COC. 

Of  the  70  contracts  awarded  after  issuance  of  a COC, 
5 were  service  contracts,  for  such  things  as  janitorial 
service  or  food  service,  and  were  not  included  in  the  first 
sub-population  group.  As  a result,  only  65  contracts 
awarded  under  COCs  were  considered  in  the  study.  Similarly, 
11  of  the  171  contracts  awarded  without  issuance  of  a COC 
were  service  contracts.  Therefore,  only  160  contracts 
awarded  without  issuance  of  a COC  were  included  in  the 
second  sub-population  group.  The  reason  these  service  con- 
tracts were  excluded  from  the  population  is  that  they  were 


awarded  to  satisfy  base-level  requirements  and  were  not 
central  procurement  contracts  Issued  by  the  ALCs. 

In  consideration  of  the  small  number  of  contracts 
awarded  under  issuance  of  a COC,  no  sample  was  taken  from 
this  sub-population  group.  Instead,  a census  measuring  the 
parameters  of  this  sub-population  group  was  utilized. 

However,  sampling  procedures  were  utilized  for  the 
second  sub -population  group.  This  procedure  consisted  of 
taking  a random  sample  from  the  remaining  160  contracts  that 
were  awarded  without  issuance  of  a COC.  Statistics  were 
then  computed  from  this  sample  and  utilized  in  estimating 
parameters  of  the  second  sub-population  group. 

Justification.  As  mentioned  in  Chapter  I,  over  one-third 
of  the  Air  Force  total  awards  to  small  business  were  made 
by  AFLC's  five  Air  Logistics  Centers.  It  can  be  shown, 
from  Table  1,  that  the  number  of  referrals  made  by  AFLC 
during  FY  77  and  78  represented,  on  the  average,  over  75 
percent  of  the  total  referrals  made  by  all  commands  within 
the  USAF.  Because  of  the  magnitude  in  number  of  referrals 
made  and  the  diversity  in  types  of  contracts  performed 
within  AFLC,  it  was  assumed  that  contract  referrals  within 
AFLC  are  representative  of  referrals  made  by  other  commands. 
This  assumption  is  necessary  in  case  inferences  are  made 
beyond  the  population  to  the  universe. 


Data  Collection 


Independent  variables.  The  data  for  the  two  Independent 
variables,  contracts  awarded  under  issuance  of  a COC  and 
contracts  referred  to  the  SBA  for  COC  action,  but  which  were 
awarded  without  issuance  of  a COC,  were  obtained  from  copies 
of  HQ  AFLC's  reports  to  HQ  USAF  under  DD-I  & L (Q)  1152, 
Certificate  of  Competency  Report. 

The  Certificate  of  Competency  Report  is  required  by 
the  ASPR,  Section  1-705. 4(c)(iii) , which  stipulates  that 
each  Service  will  submit  a quarterly  report  to  DoD  on  indi- 
vidual cases  referred  to  the  SBA  for  COC  determination.  For 
each  case,  the  report  must  give  the  name  of  the  contractor, 
a short  description  of  the  item  or  service  being  contracted 
for,  the  solicitation  number,  the  dollar  value  of  the  result- 
ant contract,  the  date  of  referral  to  the  SBA,  and  the  final 
disposition  of  the  case.  Each  Air  Force  major  command  is 
required  to  submit  a report  of  individual  cases  within  the 
command  to  the  Directorate  of  Contracting  and  Acquisition 
Policy,  HQ  USAF,  where  the  information  is  consolidated  into 
the  Air  Force  report  to  DoD  (I8il-l7), 

As  mentioned  earlier,  the  number  of  contracts 
awarded  under  issuance  of  a COC  was  relatively  small. 
Therefore,  the  information  required  for  this  independent 
variable  was  obtained  through  a census. 

The  specific  contracts  comprising  the  second  inde- 
pendent variable  were  obtained  from  a sampling  process. 
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The  sampling  process  utilized  was  that  of  random  sampling 
and  was  conducted  In  the  following  manner. 

The  solicitation  numbers  of  contracts  referred  to 
the  SBA  for  COC  action,  but  which  were  awarded  without  issu- 
ance of  a COC,  were  obtained  from  the  Certificate  of  Compe- 
tency Reports  from  each  ALC,  These  solicitation  numbers 
were  then  assigned  a unique  reference  number  and  pooled 
together  to  form  one  group.  A sample  was  then  selected 
using  a random  entry  into  a table  of  random  numbers.  This 
sample  was  then  utilized  as  the  basis  for  computing  statis- 
tics concerning  contracts  awarded  without  issuance  of  a 
COC. 

The  determination  of  the  appropriate  sample  size 
utilized  in  computing  the  necessary  statistics  was  an 
important  part  of  the  sampling  process.  The  researchers 
determined  the  appropriate  sample  size  by  utilizing  the 
formula  (4 « 148-153) i 


P(l-P)  . N^n 
J n N-l 


The  data  utilized  in  calculating  the  sample  size 
were  as  follows i 

+ 0.10  **  sampling  error  or  the  difference  between 
the  results  obtained  from  a sample  sta- 
tistic and  the  result  which  would  have 
been  obtained  from  the  population  parameter 
(9*300).  In  other  words,  this  is  the 
desired  interval  range  within  which  the 
population  proportion,  Tf  , was  expected  to 
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lie  (4il51),  This  range  was  arbitrarily 
set  by  the  researchers. 

1«96  CT  = 95  percent  confidence  level  for  estimating 
p the  interval  within  which  the  population 

Proportion,  Tf  , could  be  expected  to  lie 
4 i 151).  This  confidence  level  was  arbi- 
trarily established  by  the  researchers. 

QT  ■ the  standard  error  of  the  proportion,  com- 
p puted  from  the  sample  (4«151»  9i300). 


P(l-P)  = measure  of  sample  dispersion  where  P 

equals  the  sample  proportion.  Since  no 
information  had  been  obtained  concerning 
the  probable  P value,  the  researchers  set 
P equal  to  .5  in  order  to  establish  a 
maximum  sample  size  (4il5l»  9»345). 

= finite  correction  factor  where  N **  popu- 
”i  lation  size  and  n = sample  size.  Utilized 
when  the  ratio  of  the  sample  size  to  the 
population  size  exceeds  5 percent  (9i320). 
For  the  purposes  of  this  research,  N was 
determined  to  equal  160. 


Using  the  data  given,  the  maximum  sample  size  required  was 
calculated  to  be. 


tP“  y***-n-M*- 

n * 60.281  ~ 61 


Dependent  variables.  The  two  variables  chosen  to  measure 
contractor  performance  were  contract  delivery  delinquencies 
and  the  initiation  of  termination  for  default  proceedings. 
Delinquent  delivery  was  chosen  because  the  researchers  felt 
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that  it  was  the  best  single  measure  of  contractor  perform- 
ance. The  reason  is  that  a wide  range  of  contractor  prob- 
lems, such  as  quality,  financial  or  management  deficiencies, 
are  manifested  in  the  delivery  performance  of  the  contrac- 
tor. The  initiation  of  default  proceedings  was  felt  to  be 
a good  measure  of  the  severity  of  the  problems  being  experi- 
enced by  the  contractor  in  fulfilling  the  terms  of  the 
contract. 

To  collect  data  on  the  two  dependent  variables,  it 
was  necessary  to  have,  for  each  contract,  both  the  contract 
number  and  the  name  of  the  company  to  which  the  contract 
was  issued.  The  ASPR  reporting  Instructions  for  the  quar- 
terly COC  reports  require  only  that  the  initial  solicitation 
number  and  the  name  of  the  low  bidder  referred  to  the  SBA 
be  included  in  the  report.  Thus,  if  the  referral  resulted 
in  an  award  of  a COC  to  the  company,  the  available  informa- 
tion consisted  of  the  solicitation  number  and  the  name  of 
the  company  which  received  the  contract.  If  a COC  was  not 
issued  and  the  contract  went  to  the  next  lowest  bidder,  the 
only  Information  available  was  the  original  solicitation 
number.  Therefore,  in  order  to  track  the  contractor  per- 
formance results,  it  was  necessary  to  convert  all  solicita- 
tion numbers  to  contract  numbers  and,  in  the  cases  where 
COCs  were  not  Issued,  to  locate  the  name  of  the  company 
which  received  the  contract  award. 
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To  find  the  required  information,  assistance  was 
requested  from  the  ALCs.  A letter  was  sent  to  each  ALC 
Small  Business  Representative,  under  the  signature  of  the 
HQ  AFLC  Executive  for  Small  Business,  Mr.  Robert  B.  Kennedy. 
The  letter  asked  that  each  addressee  provide  the  contract 
number  corresponding  to  each  solicitation  number  and,  where 
necessary,  the  name  of  the  company  receiving  the  contract. 

A copy  of  the  letter  is  at  Appendix  A. 

While  the  information  on  the  contract  numbers  and 
contractor  names  was  being  collected,  an  examination  of  the 
possible  sources  of  contractor  performance  information  was 
made.  Initially,  four  possible  sources  of  data  on  contrac- 
tor defaults  and  late  deliveries  had  been  identified. 

These  were  the  AFLC  J041  Acquisition  and  Due-In  System 
(ADIS) i the  Contractor  Responsibility  Review  Program  (CRRP) 
files  maintained  at  the  ALCsi  the  individual  contract  files, 
also  kept  at  the  ALCsj  and  the  contractor  performance  files 
maintained  by  the  Defense  Contract  Administration  Service 
(DCAS)  offices  responsible  for  administering  the  contracts. 

A major  portion  of  J041  ADIS  reporting  is  the  main- 
tenance of  data  related  to  procurement  documents.  The  pro- 
duction status  reporting  portion  of  ADIS  is  used  to  identify 
and  track  all  active  ALC -awarded  central  procurement  con- 
tracts. As  an  aid  to  management  use  of  the  output  of  this 
system,  "contractors'  production  performance  history  is 
established  and  maintained  locally  within  J041  for  each 
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contractor  having  been  awarded  a contract  by  the  ALC 
Cl7tl-l]."  This  data  source  was  preferred,  since  the 
information  in  the  J041  system  could  be  accessed  at  AFLC 
Headquarters  at  Wright -Patterson  AFB.  The  J041  data  pro- 
ducts were  examined  to  see  if  the  contractor  performance 
data  captured  in  the  system  contained  the  specific  data 
elements  necessary  to  establish  whether  or  not  contractor 
default  or  late  delivery  occurred  in  each  of  the  contracts 

being  examined.  Unfortunately,  this  data  source  proved  to 

% 

be  unacceptable  when  it  was  found  that  delivery  information 
was  recorded  in  the  system  by  contractor  name  and  contract 
line  item  number  only.  The  system  provided  no  way  to 
readily  relate  performance  data  to  a specific  contract. 

The  second  potential  data  source,  the  CRRP  files, 
is  maintained  at  each  ALC.  A separate  file  is  maintained 
for  each  company  from  which  the  ALC  purchases  central  pro- 
curement items.  The  files  are  designed  to  provide  perform- 
ance data  on  specific  contractors  to  enable  contracting 
officers  to  determine  more  accurately  the  responsibility  of 
prospective  contractors  (19il).  According  to  the  imple- 
menting directive  for  the  CRRP  program,  each  file  is  to 
contain  such  contractor  performance  indicators  "as  contract 
delivery  delinquencies,  . . . default  terminations  £l9il]." 
As  with  the  J041  system,  however,  this  source  of  data 
proved  to  be  inadequate.  A review  of  the  CRRP  files  main- 
tained at  one  ALC  revealed  that,  as  with  the  J041, 
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performance  Information  was  not  maintained  with  respect  to 
individual  contract  numbers.  Additionally,  in  many  cases, 
information  was  not  maintained  for  longer  than  the  previous 
six  months. 

The  two  remaining  sources  of  information,  the  con- 
tract files  maintained  at  the  ALCs  and  the  contractor  per- 
formance files  maintained  by  the  DCAS  contract  administra- 
tion offices,  were  found  to  be  adequate  for  obtaining  the 
required  data. 

As  the  initial  step  in  obtaining  performance  data 
from  the  contract  files  at  the  ALCs,  the  researchers  visited 
San  Antonio  ALC  in  San  Antonio,  Texas.  There  was  a twofold 
purpose  in  the  visit.  First,  since  more  contracts  under 
study  were  awarded  by  San  Antonio  ALC  than  by  any  other 
single  ALC,  it  was  felt  that  if  the  researchers  performed 
the  data  search  there  it  would  significantly  ease  the  data 
collection  burden  on  the  ALCs.  Second,  it  was  felt  that 
the  knowledge  gained  from  a personal  review  of  the  contract 
files  at  one  ALC  would  aid  in  formulating  the  questions 
asked  of  the  other  ALCs.  After  the  performance  data  were 
gathered  at  San  Antonio  ALC,  a letter  was  sent  to  each  of 
the  other  ALCs  requesting  that  the  following  questions  be 
answered  for  each  contract  identified. 

1.  Did  the  contractor  deliver  the  product  in 
accordance  with  the  scheduled  delivery  dates/shipping 
instruction  specified  in  the  original  contract? 
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2.  If  the  answer  to  Question  1 Is  "No,"  was  the 
late  delivery  the  fault  of  the  contractor  or  the  Government? 

3.  What  was  the  extent  of  the  late  delivery? 

4.  Were  termination  for  default  proceedings  initi- 
ated for  this  contract? 

A copy  of  the  letter  sent  to  the  ALCs  is  at  Appendix  B. 

The  contractor  performance  files  maintained  by  DCAS 
are  kept  at  the  DCAS  Management  Area  (DCASMA)  office  within 
each  region  which  actually  administered  the  particular  con- 
tract. One  of  the  key  records  kept  in  each  file  is  the 
Production  Schedule  Completion  Record  (PSCR) . This  ADP 
report  is  maintained  by  contractor  and  contains  performance 
information  on  every  contract  that  the  contractor  completes 
within  each  monthly  reporting  period.  The  monthly  reports 
are  summarized  in  quarterly  reports  and  these  quarterly 
reports  are  normally  maintained  for  a period  of  one  year 
(21 i 88).  The  information  contained  in  these  reports  includes, 
by  contract,  whether  or  not  the  contract  was  completed  on 
schedule  and,  if  not,  the  reason  for  delay,  the  length  of 
delay,  and  whether  or  not  termination  for  default  was  made. 

To  collect  information  from  this  source,  the  contracts  were 
grouped  by  the  DCASMAs  to  which  the  individual  contractors 
were  assigned.  Then,  each  DCASMA  was  queried  by  telephone 
for  performance  information  on  the  specific  contracts  per- 
taining to  that  office. 


The  fact  that  the  Information  requested  from  the 
ALC  contract  files  and  the  DCASMA  contractor  performance 
files  was  essentially  the  same  may  make  it  appear  that 
duplication  of  effort  occurred  in  the  data  collection. 

There  was,  however,  a reason  for  using  both  sources.  Since 
the  DCASMA  information  is  generally  not  maintained  for  more 
than  one  year,  it  was  felt  that  the  ALC  contract  files  would 
be  the  best  source  of  information  on  the  older  contracts. 

On  the  other  hand,  the  DCASMA  offices,  being  responsible 
for  actual  contract  administration,  were  likely  to  have  the 
most  current  information  on  the  more  recent  contracts.  For 
those  contracts  on  which  we  received  data  from  both  sources, 
the  duplication  of  information  provided  a good  cross-check 
to  examine  the  reporting  consistency  of  the  information 
sources.  In  these  cases,  the  information  received  was 
found  to  be  virtually  identical. 

Answering  the  Research  Questions 

The  research  questions  to  be  answered  are  restated 
as  follows i 

1.  To  what  extent  has  poor  contractor  performance, 
with  respect  to  late  deliveries  and  contractor  default, 
been  evident  in  cases  in  which  COCs  have  been  issued? 

2.  To  what  extent  has  poor  contractor  performance, 
with  respect  to  late  deliveries  and  contractor  default, 
been  evident  in  cases  in  which  referrals  to  the  SBA  were 
made,  but  no  COCs  were  issued? 
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3.  Is  there  a significant  difference  in  contractor 
performance  under  each  of  the  two  categories  mentioned 
above? 

Research  questions  1 and  2 were  answered  directly 
based  on  information  obtained  from  the  five  ALCs  and  the 
DCAS  contract  administration  offices.  Research  question  3 
was  answered  by  testing  the  following  hypotheses i 

A.  There  is  no  significant  difference  between  the 
proportion  of  late  deliveries  among  contracts  performed 
under  issuance  of  a COC  as  compared  to  the  proportion  of 
late  deliveries  among  contracts  performed  without  issuance 
of  a COC. 

B.  There  is  no  significant  difference  between  the 
proportion  of  contractor  defaults  among  contracts  performed 
under  issuance  of  a COC  as  compared  to  the  proportion  of 
contractor  defaults  among  contracts  performed  without  issu- 
ance of  a COC. 

The  questions  analyzed  were  whether  the  differences 
between  the  two  computed  proportions  were  statistically 
discernible.  That  is,  assuming  that  the  data  were  suffi- 
cient to  allow  only  a small  sampling  error,  was  there  a 
statistically  discernible  difference  between  contracts  per- 
formed under  a COC  and  those  contracts  performed  without 
issuance  of  a COC? 

The  measurement  scale  (data  level)  of  the  independ- 
ent random  variable  was  nominal,  and  its  classification 
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category  (valu?  level)  was  discrete  dichotomous.  That  is 
to  say,  the  independent  random  variable  selected  was  classi- 
fied as  either  contracts  awarded  under  issuance  of  a COC  or 
contracts  awarded  without  issuance  of  a COC. 

4 The  measurement  scale  and  classification  category 
of  the  dependent  random  variables  were  also  nominal  and 
dichotomous,  respectively.  That  is,  the  contractor's  per- 
formance was  either  satisfactory  or  unsatisfactory.  In 
terms  of  delivery  schedule,  the  delivery  was  either  on-time 
(satisfactory)  or  late  (unsatisfactory).  Contractor  per- 
formance in  terms  of  default,  for  the  purposes  of  this 
research,  signified  whether  or  not  the  performance  under 
the  terms  of  the  contract  was  of  such  nature  that  the  Air 
Force  initiated  termination  for  default  proceedings  against 
the  contractor. 

Classifying  the  dependent  variables  in  this  manner 
lent  itself  to  the  use  of  proportions  as  an  aid  in  perform- 
ing statistical  tests  on  the  data.  Proportions  are  nothing 
more  than  disguised  averages  of  0-1  variables  where  0 and  1 
represent  satisfactory  and  unsatisfactory  performance, 
respectively.  As  disguised  averages  of  0-1  variables,  pro- 
portions can  be  handled  easily  with  the  general  theory  of 
sampling  and  utilized  in  the  computation  of  confidence 
intervals  for  purposes  of  testing  hypotheses  ( 22 i 166-170, 


Unless  otherwise  specifically  noted,  the  remainder 
of  this  chapter  consolidates  any  reference  to  the  two 
dependent  variables,  late  deliveries  and  contractor  default, 
into  a single  dependent  variable- -contractor  performance- - 
for  reasons  of  brevity  in  explaining  the  statistical  test 
that  was  utilized. 

As  mentioned  earlier,  a census  was  taken  of  the 
contracts  awarded  under  issuance  of  a COC.  As  a result, 
the  proportion  of  contracts  performed  unsatisfactorily 
within  this  group  represents  a parameter  of  this  sub- 
population and  is  labeled  If  . 

The  proportion  of  contracts  performed  unsatisfac- 
torily with  the  second  sub-population  group,  contracts 
awarded  without  a COC,  was  computed  from  a sample  of  that 
group.  This  proportion,  labeled  P,  is  a statistic  since  it 
describes  a characteristic  of  a sample  instead  of  a popula- 
tion (9 i 299). 

As  previously  noted,  the  hypothesis  that  was  tested 
in  answering  research  question  3 was  that  there  is  no  sig- 
nificant difference  between  the  proportion  of  late 
deliveries /contractor  defaults  among  contracts  performed 
under  issuance  of  a COC  as  compared  to  the  proportion  of 
late  deliveries/contractor  defaults  among  contracts  per- 
formed without  issuance  of  a COC.  In  other  words, 

H^i  Tf  with  COC  - P without  COC  a 0 

Hx»  Tf  with  COC  - P without  COC  f 0 
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The  test  of  the  null  hypothesis  was  conducted  by 
constructing  a confidence  interval  around  P to  see  if  it 
included  It  (22«243).  The  construction  of  this  confidence 
interval  was  accomplished  by  the  use  of  the  following 
formula  (9«340j  22i223)t 


where 


It  = proportion  of  contracts  performed  unsatis- 
factorily within  the  sub-population  group — 
contracts  with  COC.  Parameter  computed  from 
a census. 


P =*  proportion  of  contracts  performed  unsatis- 
factorily within  a sample  from  the  sub- 
population  group- -contracts  without  COC. 
Statistic  computed  from  a sample. 

t 025  ° critical  value  leaving  2\  percent  of  the 

probability  in  the  upper  tail.  Based  on  a 
5 percent  significance  level  arbitrarily 
established  by  the  researchers,  t.025  is 
utilized  instead  of  Z,o25  since  sample  size 
is  only  moderately  large  (22 i 224). 

j"?  = finite  correction  factor  for  finite  popu- 
lations,  where  N = population  size  and  n = 
sample  size.  Utilized  when  the  ratio  of 
the  sample  size  to  the  population  size 
exceeds  5 percent  (9i320),  For  the  purposes 
of  this  research,  the  population  size  was 
160  and  the  expected  sample  size  was  61, 


.381,  which  is  obviously  greater 
than  .05. 


Once  the  confidence  interval  was  constructed,  it  was 
used  immediately  to  test  the  null  hypothesis.  If  the  param- 
eter Tf  was  found  to  lie  outside  the  confidence  interval 
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A Summary  List  of  Assumptions 

1.  Contract  referrals  within  AFLC  are  representa- 
tive of  referrals  made  by  other  Air  Force  commands.  This 
assumption  is  necessary  in  case  inferences  are  made  beyond 
the  population  to  the  universe. 

2.  The  time  period  covered  by  this  research  is  of 
sufficient  duration  to  provide  valid  test  results. 

3.  The  administration  of  contracts  awarded  under 
issuance  of  a COC  is  not  significantly  different  than  the 
administration  of  contracts  awarded  without  issuance  of  a 
COC,  with  respect  to  insuring  on-time  deliveries. 

4.  The  data  loss  which  occurs  during  data  collec- 
tion and  analysis  does  not  bias  the  results  of  the  tests 
performed. 

A Summary  List  of  Limitations 

1.  This  study  does  not  analyze  causes  of  poor 
contractor  performance,  but  only  determines  whether  it  was 
present. 

2,  This  study  does  not  address  the  criteria  used 
to  justify  referral  of  a low  bidder  to  the  SBA,  nor  does  it 
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examine  the  criteria  used  by  the  SBA  to  determine  qualifica- 
tion for  COC  issuance. 

3.  The  conclusions  of  this  research  apply  only  to 
central  procurement  contracts  issued  by  the  Air  Logistics 
Centers . 

The  next  chapter  covers  data  presentation,  analysis, 
and  findings. 
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CHAPTER  IV 

DATA  PRESENTATION,  ANALYSIS,  AND  FINDINGS 

This  chapter  presents  the  data  collected  and  states 
the  findings  resulting  from  statistical  analysis  of  the 
data. 

Data  Presentation  and  Analysis 

As  mentioned  in  Chapter  III,  Methodology,  the 
Initial  data  for  this  research  project  were  obtained  from 
copies  of  HQ  AFLC  reports  to  HQ  USAF  under  DD-I  fit  L (Q) 
1152,  Certificate  of  Competency  Report.  A detailed  summary 
of  this  data  is  presented  in  Table  2.  As  shown  in  Table  2, 
the  resulting  sub-population  groups  utilized  for  this 
research  project  consisted  of  65  contracts  awarded  under 
issuance  of  a COC  and  160  contracts  awarded  without  issu- 
ance of  a COC. 

The  overall  distribution  of  each  sub-population 
group,  by  ALC  and  fiscal  year,  is  given  in  Table  3a. 

Tables  3b  and  3c  summarize  these  data  into  frequency  dis- 
tributions within  and  among  fiscal  years,  respectively. 

The  data  contained  in  these  tables  indicate  that  the  per- 
centage of  contracts  awarded  under  issuance  of  a COC,  as 
compared  to  the  percentage  of  those  contracts  awarded  with- 
out Issuance  of  a COC,  has  declined  since  Fiscal  Year  1976. 
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cancelled  purchase  requests,  and  solicitations  which  were  re fern 
COC  action  but  which  were  pending  at  the  time  of  data  collection 


. 


[ 


Table  3a 


Distribution  of  Sub-population  Groups 
by  ALC  and  Fiscal  Year 


Sub-population 

Group 

ALC 

FY  76 

FY  7T 

FY  77 

FY  78 

Total 

SA-ALC 

11 

2 

4 

7 

24 

SM-ALC 

5 

1 

9 

2 

8 

COC 

OC-ALC 

6 

1 

1 

5 

13 

00 -ALC 

1 

2 

4 

2 

9 

WR-ALC 

6 

2 

2 

1 

11 

Total 

29 

8 

11 

17 

65 

SA-ALC 

20 

2 

10 

23 

55 

SM-ALC 

7 

9 

9 

9 

25 

Non-COC 

OC-ALC 

5 

2 

10 

18 

35 

00 -ALC 

9 

4 

2 

9 

24 

WR-ALC 

7 

1 

7 

6 

21 

Total 

48 

9 

38 

65 

160 

SA-ALC 

31 

4 

14 

30 

79 

SM-ALC 

12 

1 

9 

11 

33 

Combined 

OC-ALC 

11 

3 

11 

23 

48 

00- ALC 

10 

6 

6 

11 

33 

WR-ALC 

13 

3 

9 

7 

32 

Total 

77 

17 

49 

82 

225 
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Frequency  Distribution  of  Sub-population  Groups  Within  Fiscal  Years 
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A detailed  analysis  of  this  apparent  trend  was  con- 
sidered beyond  the  scope  of  this  research  project}  there- 
fore, these  data  are  presented  for  informational  purposes 
only. 

Other  data  initially  collected  from  the  quarterly 
Certificate  of  Competency  Reports  are  contained  in  Table  4a. 
This  table  displays  the  overall  distribution  of  each  sub- 
population group  by  ALC  and  contract  dollar  value.  The  data 
in  Table  4a  were  then  summarized  into  frequency  distribu- 
tions of  sub-population  groups  within  and  among  contract 
dollar  value  categories  and  are  presented  in  Tables  4b  and 
4c,  respectively. 

Table  4c  indicates  that  the  majority  of  contracts 
awarded  within  each  sub-population  group  was  distributed 
among  the  lower  dollar  value  categories  (i.e.,  less  than 
$100,000).  Table  4b  indicates  that  as  the  dollar  value  of 
the  contract  increased  (up  to  $1,000,000)  the  percentage  of 
contracts  awarded  under  issuance  of  a COC  increased,  as 
compared  to  the  percentage  of  contracts  awarded  without 
issuance  of  a COC. 

Again,  these  data  are  presented  for  informational 
purposes  only,  as  the  researchers  considered  an  in-depth 
analysis  to  be  beyond  the  specific  objectives  of  this  study. 

In  order  to  complete  the  objectives  of  this  study, 
an  attempt  was  made  to  collect  performance  data  on  specific 
contracts  within  each  sub-population  group.  As  explained 
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Table  4a 
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Total  122  37  47  8 11  225 
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in  Chapter  III,  a census  totaling  65  contracts  was  taken 
from  the  sub-population  group  consisting  of  contracts 
awarded  under  issuance  of  a COC.  From  this  group,  complete 
information  was  collected  for  46  contracts  providing  a data 
yield  rate  of  70.8  percent. 

From  the  160  contracts  comprising  the  second  sub- 
population group,  contracts  awarded  without  issuance  of  a 
COC,  a random  sample  of  61  contracts  was  selected.  Complete 
information  was  obtained  for  44  contracts  from  this  sample, 
providing  a 72.1  percent  data  yield  rate. 

The  reduction  in  the  number  of  contracts  for  which 
complete  information  was  collected  was  due  to  one  or  more 
of  the  following  reasons t 

1.  The  inability  to  convert  solicitation  numbers  of 
older  contracts  into  contract  numbers. 

2.  The  purchase  request  being  cancelled  after  a 
contract  had  been  referred  to  the  SBA. 

3.  The  inability  to  locate  records  on  selected 
contracts • 

4.  Delivery  requirements  of  later  contracts  being 
not  yet  due,  i.e.,  current  contracts. 

A summary  of  this  information  is  presented  in 
Table  5,  Results  of  Contract  Information  Requests. 

Earlier  calculations  (Chapter  III)  utilizing  the 
formula  for  computing  maximum  required  sample  size 
(4.148-153), 


indicated  that  a sample  size  of  61  would  provide  for  a max- 
imum sampling  error  of  - 10  percent.  Working  with  a sample 


size  of  44  contracts  for  which  complete  information  was 

I 

collected,  the  researchers  calculated  the  revised  maximum 
sampling  error  to  be 

ss 

Sampling  Error  (SE)  = .126  or  * 

Since  this  percentage  represented  the  maximum  sam- 
pling error  that  could  be  expected,  the  researchers  felt 
that  the  slight  increase  of  2.6  percent  was  acceptable. 

As  a result  of  the  time  constraint  involved  in  com- 
pleting this  study  and  the  feeling  that  the  smaller  sample 
was  still  an  unbiased  representation  of  its  sub-population 
group,  no  additional  random  samples  were  taken.  The 
researchers  therefore  concentrated  their  effort  toward  the 
analysis  of  the  resulting  contract  information  collected. 

In  analyzing  the  delivery  performance  data  of 
selected  contracts  from  each  sub-population  group,  it  was 
discovered  that  four  contracts  from  each  group  were  delin- 
quent as  the  result  of  government- caused  delays.  Since  the 
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individual  contractor's  delivery  performance  could  not  be 
determined  on  these  contracts,  they  were  subsequently 
excluded  from  further  analysis.  This  action  essentially 
reduced  the  number  of  COC  and  non-COC  contracts  usable  for 
analysis  to  42  and  40  contracts,  respectively. 

Of  the  42  contracts  awarded  under  issuance  of  a COC, 
23  encountered  delivery  delays  resulting  in  a delinquency 
rate  of  54.8  percent.  The  delinquency  rate  for  the  40  con- 
tracts awarded  without  issuance  of  a COC  was  found  to  be 
higher  at  62.5  percent.  This  information  is  summarized  in 
Table  6,  Delivery  Performance  for  Contracts  Included  in 


Study. 


Because  the  number  of  non-COC  contracts  usable  for 


analysis  had  been  reduced  from  44  to  40,  the  researchers 
felt  that  a recalculation  of  possible  sampling  error  was  in 
order.  Following  the  procedures  described  above,  but  using 
the  actual  delinquency  rate  encountered,  the  researchers 
calculated  the  maximum  sampling  error  that  could  be  expected 


to  be, 


SE 

05 


625(1-. 625 


.1303  or  13.0%. 
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Since  this  sampling  error  represented  only  a 0.4  per- 
cent increase  from  the  maximum  sampling  error  which  could  be 


expected  from  a sample  of  44  contracts,  or  3.0  percent 


increase  from  the  original  sample  of  61  contracts,  the 
researchers  felt  that  this  sampling  error  was  acceptable  and 
would  not  significantly  bias  the  overall  results. 

One  by-product  of  obtaining  data  on  delivery  per- 
formance was  that  of  receiving  information  on  the  length  of 
delays  encountered  among  the  delinquent  contracts.  This 
information  is  summarized  in  Table  7.  It  can  be  seen  from 
the  information  presented  in  Table  7 that  of  the  contracts 
which  were  awarded  under  issuance  of  a COC,  the  percentage 
which  was  delinquent  for  60  days  or  more  was  8,6  percent 
higher  than  the  same  category  of  contracts  awarded  without 
issuance  of  a COC.  This  difference  in  percentages  is  dis- 


cussed in  Chapter  V. 


Table  7 

Duration  of  Delays 


1-30 


31-60 


Non- COC 


Number  Late 
7.  of  Total 


Number  Late 


30,4  13.0 


7.  of  Total  36.0  16.0 


Over  60 


56.6 


48.0 


Totals 


1007. 


100% 


The  final  data  presentation  and  analysis  conducted 
for  this  study  was  in  the  area  of  terminations  for  default. 
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The  objective  of  this  portion  of  the  research  project  was 
to  determine  to  what  extent  contractor  defaults  have  been 
evident  in  contracts  awarded  under  issuance  of  a COC  as 
compared  to  those  contracts  awarded  without  issuance  of  a 
COC. 

Data  collected  for  those  42  COC  contracts  which  were 


considered  usable  for  analysis  indicated  that  there  were 
four  contracts  (9.5  percent)  in  which  termination  for 
default  proceedings  had  been  initiated.  As  Table  8 indi- 
cates, this  9.5  percent  appears  to  be  in  sharp  contrast  with 
the  2.5  percent  non-COC  contracts  for  which  termination  for 
default  proceedings  were  initiated.  The  statistical  signi- 
ficance of  this  apparent  difference  is  elaborated  upon  more 
fully  in  the  next  section. 

Table  8 

Initiations  of  Termination  for  Default 


Total 

Contracts  Usable 

Terminations 

Percent 

for  Analysis 

for  Default 

of  Total 

COC 

42 

4 

9.5 

Non-COC 

40 

1 

2.5 

Findings 

As  explained  in  Chapter  III,  Methodology,  the  first 
research  hypothesis  tested  wasi 
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There  is  no  significant  difference  between 
the  proportion  of  late  deliveries  among 
contracts  performed  under  Issuance  of  a 
COC  as  compared  to  the  proportion  of  late 
deliveries  among  contracts  performed  with- 
out Issuance  of  a COC.  In  other  words, 

Hjji  TC  with  COC  - P without  COC  = 0 

H1»  Tf  with  COC  - P without  COC  5*  0 

The  results  of  the  data  analysis  on  delivery  delin 

quencies,  taken  from  Table  6,  are  displayed  in  Table  9. 

Table  9 

Contract  Delinquencies 


Contracts  Usable  Number  on  Number  Percent 
for  Analysis  Schedule  Delinquent  Delinquent 


COC 

42 

19 

23 

54.8 

Non-COC 

40 

15 

25 

62.5 

The  formula  utilized  to  construct  the  95  percent 
confidence  interval  around  P (proportion  of  non- COC  con- 
tracts with  late  deliveries)  to  see  if  it  included  T(  (pro- 
portion of  COC  contracts  with  late  deliveries)  was,  as 
described  in  Chapter  IIIi 


P^t 


.025 


fP(l-P)  . N^n 
n N-l 


3 


Using  this  formula,  the  95  percent  confidence 
interval  around  P was  calculated  to  bei 
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Since  the  parameter  Tft  (.548)  was  found  to  lie 
within  the  confidence  interval  established  around  the 
statistic  P (.491  to  .759)  we  cannot  reject  the  null 
hypothesis,  H^t  If  with  COC  - P without  COC  **  0.  That  is 
to  say,  there  appears  to  be  no  significant  statistical  dif- 
ference in  the  proportion  of  delivery  delinquencies  between 
those  contracts  awarded  under  issuance  of  a COC  as  compared 
to  those  contracts  awarded  without  issuance  of  a Certificate 
of  Competency. 

The  second  research  hypothesis  tested  wasi 

There  is  no  significant  difference  between 
the  proportion  of  contractor  defaults  among 
contracts  performed  under  issuance  of  a 
COC  as  compared  to  the  proportion  of  con- 
tractor defaults  among  contracts  performed 
without  issuance  of  a COC.  That  is  to  say, 

H0i  Tf  with  COC  - P without  COC  - 0 

H^i  Tf  with  COC  - P without  COC  ^ 0 

The  results  of  the  data  analysis  on  terminations  for 

default,  taken  from  Table  8,  are  reproduced  in  Table  10. 

Table  10 

Initiations  of  Termination  for  Default 

Total  Contracts  Usable  Terminations  Percent 
for  Analysis  for  Default  of  Total 

COC  42  4 9.5 

Non- COC 40 1 2.5 
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Following  the  same  procedures  initially  described 
in  Chapter  III  and  utilized  above  for  delivery  delinquen- 
cies, the  95  percent  confidence  interval  around  P was  cal- 
culated to  be  i 


C.I. 


C.I. 

C.I. 


.025  t 2.021 
.025  1 .043 


/ 


.025(1-. 025) 
40 


160-40 

160-1 


-0.018  lower  limit  and  0.068  upper  limit 


In  this  test  the  parameter  IT  (.095)  was  found  to 
lie  outside  of  the  confidence  interval  established  around 
the  statistic  P (-0.018  to  0.068),  Therefore,  the 
researchers  must  reject  the  null  hypothesis,  H0i  It  with 
COC  - P without  COC  =0.  In  other  words,  there  appears  to 
be  a significant  statistical  difference  in  the  proportion 
of  terminations  for  default  between  those  contracts  awarded 
under  the  issuance  of  a COC  as  compared  to  those  contracts 
awarded  without  issuance  of  a Certificate  of  Competency. 

The  conclusions  that  may  be  drawn  from  the  above 
described  test  results  are  elaborated  upon  more  fully  in 
Chapter  V,  Summary,  Conclusions,  and  Recommendations. 

The  following  chapter  presents  a research  summary, 
study  conclusions,  and  recommendations  for  further  research. 


CHAPTER  V 


SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

This  chapter  presents  a summary  of  the  research 
accomplished,  describes  the  conclusions  drawn  from  the 
findings,  and  offers  recommendations  for  further  research. 

Summary 

The  objective  of  this  research  was  to  examine  con- 
tract information  from  the  five  ALCs  to  determine  if  CGC 
procedures  have  an  adverse  impact  on  ALC  contracting  with 
respect  to  contractor  performance.  To  satisfy  this  objec- 
tive, contractor  performance  data  were  obtained  from  the 
ALCs  and  the  administering  DCASMAs  to  answer  the  following 
questions i 

1.  To  what  extent  has  poor  contractor  performance, 
with  respect  to  late  deliveries  and  contractor  default,  been 
evident  in  cases  in  which  COCs  have  been  issued? 

2.  To  what  extent  has  poor  contractor  performance, 
with  respect  to  late  deliveries  and  contractor  default,  been 
evident  in  cases  in  which  referrals  to  the  SBA  were  made, 
but  no  COCs  were  issued? 

3.  Is  there  a significant  difference  in  contractor 
performance  under  each  of  the  two  categories  mentioned 
above? 


For  the  purposes  of  data  analysis,  contractor  late 
delivery  was  defined  as  a delinquent  delivery  regardless  of 
the  length  of  delay.  Information  on  the  length  of  delin- 
quencies was  collected,  however,  and  Is  discussed  as  a cor- 
ollary finding  in  the  next  section  of  this  chapter.  Con- 
tractor default  was  measured  by  whether  or  not  termination 
for  default  proceedings  were  Initiated  against  the  contrac- 
tor. 

Performance  data  covering  Fiscal  Years  1976  through 
1978  were  gathered  on  42  COC  and  40  non-COC  contracts. 

They  were  analyzed  statistically  with  the  following  find- 
ings! 

1.  The  hypothesis  that  there  Is  no  significant 
difference  In  contractor  performance  with  irespect  to  late 
deliveries  between  COC  and  non-COC  cases  could  not  be 
rejected.  The  data  showed  that  of  the  42  COC  contracts 
examined,  23,  or  54.8  percent,  Involved  late  deliveries. 

Of  the  40  non-COC  contracts,  25,  or  62.5  percent,  were 
delinquent. 

2.  The  hypothesis  that  there  is  no  significant 
difference  between  COC  and  non-COC  cases  with  respect  to 
default  initiation  was  rejected.  This  finding,  that  there 
was  a significant  difference  in  terms  of  default,  is  based 
on  the  Initiation  of  default  proceedings  in  four  COC  cases 
versus  only  one  default  initiation  in  the  non-COC  cases. 


Conclusions 

The  above  findings  appear  to  support  a conclusion 
that,  although  there  is  no  significant  difference  in  impact 
on  the  ALCs  between  COC  and  non-COC  cases  in  terms  of  late 
deliveries,  there  is  a significant  difference  in  terms  of 
the  seriousness  of  the  problems  as  measured  by  default  ini- 
tiation. It  is  not  possible  to  make  any  resultant  infer- 
ences as  to  the  quality  of  the  SBA's  COC  Program,  however, 
unless  several  factors  which  may  have  influenced  the  per- 
formance results  are  examined. 

With  respect  to  the  number  of  delinquent  deliveries, 
the  findings  showed  no  significant  difference  between  per- 
formance under  COCs  and  performance  in  the  non-COC  cases. 
Since  COCs  are  only  Issued  in  cases  where  the  contracting 
officer  has  doubts  about  the  contractor's  ability  to  com- 
plete the  contract  satisfactorily,  the  researchers  intui- 
tively felt  that  it  was  likely  that  delivery  performance 
would  be  worse  under  COC  issuance. 

The  finding  that  there  is  no  significant  difference 
in  the  number  of  late  deliveries  could  possibly  be  attrib- 
uted to  positive  influences  of  the  SBA's  COC  Program. 

First,  the  knowledge  that  they  were  performing  under  a COC 
may  have  motivated  those  companies  to  expend  extra  effort 
in  order  to  overcome  possible  shortcomings.  Second,  there 
is  a possibility  that  the  SBA  may  have  provided  needed 
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post-award  guidance  and  assistance  to  those  companies  to 
which  it  awarded  COCs. 

On  the  other  hand,  there  is  a possibility  that  the 
ALC  contracting  officers  and  the  DCAS  contract  administra- 
tors may  have  had  to  spend  extra  time  and  effort,  at  a cost 
to  the  Government,  to  ensure  that  the  COC  contracts  were 
completed  on  time.  A suggested  topic  for  further  research 
in  this  area  appears  in  the  next  section  of  this  chapter. 
Another  possible  negative  factor  is  that  the  number  of  con- 
tract modifications  to  permit  delivery  extensions  may  have 
been  greater  for  the  COC  cases  than  for  the  non-COC  cases. 

A determination  of  the  extent  to  which  the  above 
influences,  both  positive  and  negative,  may  have  applied  was 
not  made,  primarily  because  it  was  beyond  the  purview  of 
this  research,  but  also  because  time  limitations  precluded 
such  an  in-depth  analysis.  Nevertheless,  it  is  concluded 
that  there  is  no  significant  difference  between  the  number 
of  late  deliveries  under  COC  contracts  and  the  number  of 
late  deliveries  where  COCs  were  not  issued. 

The  severity  of  the  performance  problems  between 
the  two  categories  of  contracts,  as  measured  by  the  initia- 
tion of  default  proceedings,  is  another  matter.  As  men- 
tioned, the  analysis  showed  a significant  difference  in  the 
number  of  default  initiations,  based  on  four  such  cases 
under  COC  contracts  and  only  one  under  the  non-COC  contracts. 


A conclusion  that  the  severity  of  contract  problems 
is  likely  to  be  greater  under  COC  issuance  is  apparently 
strengthened  by  the  corollary  findings  on  the  length  of 
delays  shown  in  Table  7.  Although  the  statistical  signifi- 
cance of  the  data  was  not  tested,  the  findings  indicate  that 
the  percentage  of  COC  contracts  in  the  "Over  60"  day  delay 
category  is  8.6  percent  higher  than  the  percentage  of  non- 
COC  contracts  in  that  category. 

In  summary,  it  is  concluded  that  the  COC  program  has 
no  apparent  impact  on  the  ALCs  in  terms  of  the  number  of 
delivery  delinquencies.  However,  with  respect  to  the 
severity  of  the  problems  encountered,  as  measured  by  default 
proceedings,  the  impact  is  greater  for  the  COC  contracts 
than  for  the  non-COC  contracts.  . 

Recommendations  for  Further  Study 

During  the  course  of  this  research,  several  topics 
for  additional  study  were  uncovered  by  the  researchers  or 
suggested  by  interested  outside  activities.  A few  of  these 
topics,  felt  to  be  most  significant  by  the  researchers,  sure 
presented  for  consideration. 

1.  The  conclusion  of  this  study  is  that  although 
there  is  no  significant  difference  in  the  number  of  late 
deliveries  between  COC  and  non-COC  contracts,  the  severity 
of  the  problems  encountered  is  greater  under  COCs.  A 
potential  area  for  further  research  is  an  examination  of 
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the  actual  dollar  costs  or  savings  to  the  Air  Force  result- 
ing from  the  COC  program.  The  study  would  Initially  entail 
a look  at  both  COC  contracts  and  comparable  non- COC  con- 
tracts in  terms  of  both  contract  administration  costs  and 
the  costs  resulting  from  inadequate  performance  and  default 
proceedings.  These  costs  would  then  be  measured  against 
the  savings  resulting  from  the  award  to  low  bidders  under 
COCs  versus  award  to  the  second  low  bidder  when  COCs  are 
not  issued.  This  kind  of  cost  tradeoff  study  would  serve 
as  a measurement  of  the  overall  cost  or  savings  to  the  Air 
Force  resulting  from  the  COC  program. 

2.  Table  3b  shows  that  the  percentage  of  referrals 
to  the  SBA  that  have  culminated  in  the  award  of  COCs  has 
been  declining  over  the  past  three  fiscal  years.  The  table 
shows  that  of  the  ALC  referrals  to  the  SBA,  the  number 
resulting  in  COCs  has  declined  from  37.7  percent  in  FY  1976 
to  20.7  percent  in  FY  1978.  A study  could  be  made  to 
determine  the  reasons  for  this  decline.  Possible  reasons 
are  that  the  SBA  is  tightening  the  criteria  it  applies  in 
evaluating  small  business  concerns  or  that  the  Air  Force  is 
relaxing  its  acceptance  criteria.  One  corollary  finding 
made  by  the  researchers  was  that  the  number  of  cases  in 
which  a firm  that  is  referred  to  the  SBA  declines  to  file 
for  a COC  has  been  growing  steadily.  For  the  period  of 
FY  1976  through  FY  1978,  it  was  found  that  in  approximately 
52  percent  of  the  instances  in  which  COCs  were  not  issued, 
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the  reason  was  that  the  firm  referred  to  the  SBA  did  not 
elect  to  file  for  a COC.  In  FY  1978  alone,  the  figure  was 
72  percent.  This  might  indicate  that  many  small  business 
firms  are  simply  "throwing  in"  bids  and  then  withdrawing 
when  challenged.  It  might,  on  the  other  hand,  indicate 
that  a toughening  of  SBA  criteria  is  causing  more  and  more 
firms  to  reconsider  applying  for  COCs. 

3.  A surprising  discovery  to  the  researchers  was 
the  high  percentage  of  delinquent  deliveries  in  both  sub- 
population groups  studied.  It  is  not  known  whether  the 
delinquency  rates  are  representative  of  all  awards  to  small 
business  or,  for  that  matter,  how  these  delinquency  rates 
compare  with  those  for  awards  to  large  business  firms.  To 
answer  questions  such  as  these,  a study  might  be  made  of 
the  extent  and  causes  of  contract  delivery  delays  on  Air 
Force  contracts,  in  general,  to  include  large  businesses. 

The  completion  of  these  additional  research  projects 
would  further  expand  our  understanding  of  contractor  per- 
formance. 
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DEPARTMENT  OF  THE  AIR  FORCE 
HEADQUARTERS  AIR  FORCE  LOGISTICS  COMMAND 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OHIO  45433 


»m.v  to 

» TTM  on  PMB 


•ubjbct.  Request  for  Contract  Information 


9 FEB  1979 


k 


TOt  OC-ALC/BC  OO-ALC/BC  SA-ALC/BC  SM-ALC/BC  WR-ALC/BC 

1.  A research  team  at  the  Air  Force  Institute  of  Technology, 
here  at  Wright-Patterson  AFB,  is  attempting  to  examine 
sample  contract  results  to  see  if  there  is  any  significant 
statistical  difference  between  contractor  performance  where 

a certificate  of  competency  (COC)  was  issued  and  performance 
where  the  contracts  were  awarded  to  other  bidders  because 
the  COC  was  not  issued. 

2.  The  researchers  have  chosen  their  sample  cases  for 
follow-up  from  the  quarterly  Air  Logistics  Center  COC  reports. 
In  many  individual  cases,  however,  the  only  identifying  in- 
formation available  is  the  name  of  the  company  referred  to 
the  SBA  and  the  solicitation  (RFP)  number.  In  order  to  track 
the  final  results,  the  research  team  will  need  the  resultant 
contract  numbers  and  the  name  of  the  contractor. 

3.  Attached  is  a list  of  cases  chosen  for  the  research 
sample.  Request  you  provide,  for  each  case,  the  contract 
number  and  the  name  of  the  firm  receiving  the  contract. 

Please  reply  to  Major  Charles  Simko,  AFIT/LSG,  Wright- 
Patterson  AFB,  OH  45433.  A pre-addressed  envelope  has  been 
included  for  your  convenience. 

4.  Your  help  in  providing  this  information  will  be  a key 
factor  in  the  successful  completion  of  the  research.  A reply 
by  23  February  1979  will  be  greatly  appreciated. 

FOR  THE  COMMANDER 

1 Atch 

Information  Request 
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DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  INSTITUTE  OF  TECHNOLOGY  (ATC) 
WRIGHT-PATTERSON  AIR  FORCE  BASE.  OHIO  45433 


MFIT  TO 

ATTN  OFI  LSGM 


Request  for  Contractor  Performance  Information 


to.  OC-ALC/PMXS 

00-ALC/PMXS 

SA-ALC/PMXS 

SM-ALC/PMXS 

WR-ALC/PMXS 

1.  Two  students  of  the  Air  Force  Institute  of  Technology 
are  attempting  to  examine  sample  contract  results  to  see  if 
there  is  any  significant  difference  between  contractor  per- 
formance under  certificate  of  competency  (COC)  issuance  and 
performance  where  COCs  were  not  issued.  The  sample  contracts 
were  chosen  from  the  quarterly  Air  Logistics  Center  COC 
reports . 

2.  We  request  your  assistance  in  determining  the  adequacy 
of  contractor  performance.  The  attached  chart  lists  the 
contract  number  and  the  name  of  the  contractor  for  each  of 
the  sample  contracts,  along  with  a few  questions  concerning 
performance.  Request  you  complete  the  chart  and  return  it, 
in  the  enclosed  pre-addressed  envelope,  to  Major  Charles 
Simko,  AFIT/LSG-79A,  Wright-Patterson  AFB,  OH  45433,  by  23 
April  1979. 

3.  Gaining  the  contractor  performance  information  will  be  • 
the  key  step  in  the  successful  completion  of  the  research. 

It  should  be  noted,  however,  that  the  research  findings  will 
not  make  reference  to  any  contractor  by  name. 

4.  This  research  project  has  been  coordinated  with  Mr. 

Robert  Kennedy,  the  Hq  AFLC  Assistant  for  Small  Business,  . 
as  well  as  with  Mr.  Frey  in  AFLC/PMM  and  Major  Huber  in 
AFLC/PMX.  If  there  are  any  questions  regarding  this  request, 
please  contact  the  undersigned  at  AUTOVON  785-5096.  Your 
support  of  our  educational  mission  is  sincerely  appreciated. 

Jack  l.  mcchesney,^  coi,  usaf  • 1 Atch 

'Assistant  Professor  of  Logistics  Information  Request 

Management 

School  of  Systems  and  Logistics 
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INFORMATION  REQUEST 
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INSTRUCTIONS 


The  answers  to  the  following  questions  are  to  be  used  in 
evaluating  a contractor's  delivery  performance  on  a speci- 
fic contract.  For  purposed  of  this  study,  a contract  is  de- 
linquent when  the  contractor,  within  his  control,  has  failed 
to  deliver  a quality  product  in  a timely  manner  (i.e.,  in 
accordance  with  scheduled  delivery  dates  specified  in  the 
original  contract) , either  incrementally  or  finally,  except 
when  corrected,  revised,  or  amended  by  a:  formal,  signed 
contract  change  order  or  modification. 

1.  Did  the  contractor  deliver  the  product  in  accordance 
with  the  scheduled  delivery  dates/shipping  instruction 
specified  in  the  original  contract?  Please  circle  the 
correct  response. 

2.  If  the  answer  to  Question  1 is  "NO",  was  the  late  de- 
livery the  fault  of  the  contractor  or  the  Government? 
What  was  the  extent  of  the  late  delivery?  Please  circle 
the  correct  responses. 

3.  Were  termination  for  default  proceedings  initiated  for 
this  contract?  Please  circle  the  correct  response. 
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CONTRACT 

CONTRACTOR 

NUMBER 

QUESTIONS 

1 

* 

2 

3 

YES/foO 

CNTRCTR/GOV 'T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/NO 

YES/*lO 

CNTRCTR/GOV ' T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/foO 

YES/ftO 

CNTRCTR/GOV' T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/ftO 

YES/foO 

CNTRCTR/GOV 'T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/fclO 

YES/NO 

CNTRCTR/GOV 'T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/ftO 

YES/NO 

CNTRCTR/GOV 'T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

V-%. 

YES/foO 

YES/NO 

CNTRCTR/GOV 'T 

L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/foO 

YES/faO 

CNTRCTR/GOV' T 

•••  • 

L.T.  30  Days/ 

30  to  60  Days/ 

G.T.  60  Days 

YES/ftO 

YES/ftO 

CNTRCTR/GOV 'T 
L.T.  30  Days/ 

30  to  60  Days/ 
G.T.  60  Days 

YES/ftO 

* L.T. 
G.T. 


"Less  Than" 
"Greater  Than" 
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